Comparison between Dynamic Programming and Reinforcement Learning: a Case Study on Maize Irrigation Management

Irrigation scheduling is an important decision problem in agriculture. The purpose of the study presented in this paper is to compare dynamic programming (DP) and reinforcement learning (RL) methods for identifying optimal starting irrigation strategies, when a limited amount of water is available for irrigation. Both of the optimization methods use the MODERATO simulator, which includes a growth simulator and an irrigation strategy simulator for maize crops, coupled with a stochastic random weather generator. The results we present illustrate for each of these methods the relations between the number of simulation runs, the size of the discretization and the quality of the approximated optimal strategy that is obtained. INTRODUCTION Irrigation scheduling in agriculture is an important decision problem that has a major effect on yield, environment and gross margin in water limited areas. Environmentalists and politicians criticise farmers who use a large amount of the available water to irrigate their crops. Applying too much water can potentially create nitrate problems in the groundwater tables. In some irrigation areas where water resources are limited, farmers may receive a fixed amount of water for the growing season or may have to restrict their irrigation flow rates at certain times. For a farmer the profitability of irrigated crops can be improved by reducing the amount of water used and optimizing the timing of application (Stockle and James, 1989). Further, increasing the utilisation of water applied reduces the likelihood of water table accessions, an environmental benefit. New irrigation scheduling approaches, not necessarily based on satisfying the full crop water requirement, but aimed at increasing the efficiency of allocated irrigation water so as to give the highest crop production with the least water use, must be developed (Kirda and Kanber, 1999). A first attempt to answer these questions is made here. The simple subproblem of deciding when to start the irrigation campaign is formulated. We then compare stochastic dynamic programming and reinforcement learning methods for identifying optimal decision rules. Both